differential fuzzer based on libFuzzer#556
Conversation
`fuzz/solidity_differential` (libFuzzer + SanitizerCoverage) — coverage-guided. Mutates inputs from SanCov edges in resolc's Rust dep graph (revive-yul, resolc, revive-llvm-context, revive-runner). Eight-template Solidity generator; `solc → EVM` direct path. Documented in book/src/developer_guide/fuzzing.md.
elle-j
left a comment
elle-j
left a comment
There was a problem hiding this comment.
First pass. Excited to see this fuzzer coming together 🙂 Will do a more thorough review once you've integrated main's changes (importantly, the addition of newyork).
Also, please make sure the Files changed do not include revive-differential-tests. 👍
| `revive` ships a coverage-guided differential fuzzer that compares the | ||
| same logical contract execution between resolc's PVM lowering and |
There was a problem hiding this comment.
Could you update this file to not split paragraphs mid-sentence (just have paragraphs on one line)? It makes it more consistent and also easier when updating the file without having to deal with certain formatting 👍
| SanitizerCoverage, so the libFuzzer mutation engine sees edges in: | ||
|
|
||
| * `revive-yul` parser | ||
| * `resolc` standard-json pipeline | ||
| * `revive-llvm-context` codegen (every lowering pattern) | ||
| * `revive-runner` / pallet-revive simulation | ||
| * `arbitrary` and the generator itself |
There was a problem hiding this comment.
Needs a revive-newyork once main is merged into this branch. (Same for revive/fuzz/README.md.)
There was a problem hiding this comment.
Agree. Also after reading Georgiy's post, there this fuzzer can be improved in many ways.
| > A SanCov-instrumented LLVM at `$LLVM_SYS_221_PREFIX` will break | ||
| > non-fuzz `cargo build` invocations: the linker needs | ||
| > `__sanitizer_cov_*` symbols that only the libFuzzer runtime | ||
| > supplies. Keep two LLVM trees if you need both: switch via | ||
| > `LLVM_SYS_221_PREFIX`. |
There was a problem hiding this comment.
Out of curiosity, what's your own workflow/setup when using both non-instrumented LLVM and the fuzzer (needing instrumented LLVM)?
There was a problem hiding this comment.
I don't have enough disk space to keep both LLVMs, so I do rebuild. Otherwise I would keep 2 LLVM versions like it's mentioned here, i.e. build LLVM once, move it to some target-llvm-1 and build another LLVM
|
|
||
| ## Divergence taxonomy | ||
|
|
||
| `Divergence` (in `crates/fuzz/src/differential.rs`) categorises every |
There was a problem hiding this comment.
Nit: Could you update the file to find wherever spellings should be updated to American spelling (e.g. categorises -> categorizes, optimiser -> optimizer, etc) for consistency.
| Compile failures used to panic the whole process via | ||
| `.expect("source should compile")` inside resolc's `test_utils`. The | ||
| harness wraps both calls in `std::panic::catch_unwind` and routes the | ||
| payload into a dedicated variant, so a generator bug doesn't poison | ||
| the whole libFuzzer run. |
| Coverage-guided fuzzer for the Solidity differential, driven by | ||
| [`cargo-fuzz`](https://github.com/rust-fuzz/cargo-fuzz) / libFuzzer. |
There was a problem hiding this comment.
Similar to my comment to the dev guide update, could this file be updated to not break lines mid-sentence (i.e. keep paragraphs on one line)?
| install-llvm-sancov: install-llvm-builder | ||
| git submodule update --init --recursive --depth 1 | ||
| CMAKE_BUILD_PARALLEL_LEVEL=$(JOBS) revive-llvm build --llvm-projects lld --llvm-projects clang --enable-sancov |
There was a problem hiding this comment.
We could consider moving git submodule update --init --recursive --depth 1 (from install-llvm-sancov and install-llvm) into install-llvm-builder instead so that it's only in one place.
| fuzz-libfuzzer: | ||
| cd fuzz && cargo +nightly fuzz run solidity_differential -- -fork=$(or $(JOBS),4) -ignore_crashes=0 |
There was a problem hiding this comment.
Could this also cd back to ../ so that we don't have to do it manually after running the command.
There was a problem hiding this comment.
Does the clean target also need an update? I saw that fuzz's README suggested some cleanup.
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
|
All alerts resolved. Learn more about Socket for GitHub. This PR previously contained dependency changes with security issues that have been resolved, removed, or ignored. |
2 participants
fuzz/solidity_differential(libFuzzer + SanitizerCoverage) —coverage-guided. Mutates inputs from SanCov edges in resolc's
Rust dep graph (revive-yul, resolc, revive-llvm-context,
revive-runner). Eight-template Solidity generator;
solc → EVMdirect path.
Documented in book/src/developer_guide/fuzzing.md.